System and method for generating and using fingerprints for integrity management

ABSTRACT

A system and method is provided for capturing and using problem fingerprints in an Information Technology (IT) infrastructure for integrity management. A fingerprint of the transaction topology in an IT infrastructure is automatically captured at various time intervals prior to the occurrence of an event leading to a problem, such as a failure, hard threshold violation, defined transaction violation or user-provided occurrence. The fingerprint provides an indication of the activity and operation of the IT infrastructure immediately preceding the problem event. The captured fingerprint is then used to monitor real-time data in the IT infrastructure operation and activity to look for activity that matches a captured fingerprint to provide an indication of a pending problems before the problems occur. When it is determined that there is sufficient probability a problem event will occur based upon real-time data matching a previously generated problem fingerprint, an alert is generated to provide sufficient notification prior to the occurrence of problem event.

TECHNICAL FIELD

This disclosure relates generally to the field of network monitoring systems and, more particularly, to a system and method for providing integrity management in a network environment by generating and using problem fingerprints.

BACKGROUND

A network consists of two or more computers or other devices that are linked in order to share resources (such as databases, servers, printers, etc.), exchange files, or allow electronic communications. The computers on a network may be linked together through a communication medium, such as cables, telephone lines, radio waves, satellites, or infrared light beams. There are many types of computer networks, including local-area networks (LANs), wide-area networks (WANs), campus-area networks (CANs), metropolitan-area networks (MANs) and home-area networks (HANs). Networks are used to communicate between devices, such as via e-mail, and to provide access to resources stored on another device, such as a server.

Most organizations possess an Information Technology (IT) infrastructure comprising the computerized networks, intelligent terminals, and accompanying applications and services people use to access, create, disseminate, and utilize digital information. The IT infrastructure also includes the equipment, software, services, and products used in storing, processing, transmitting, and displaying all forms of information. Organizations are increasingly dependent on their IT infrastructure for all of their essential business processes, which often depend on software, hardware, networks and data systems working together with full integrity to provide business functionality to external and internal users alike. Increasingly, “online” business processes are both critical to a company's well-being and based on distributed IT infrastructures of mounting complexity. The scope of this infrastructure might even extend beyond the organization's boundaries into the infrastructures of partners and providers of managed services.

IT infrastructures are typically not the centralized, well-understood operations characteristic of the days of mainframes, architected and tested as a whole down to the last possible race condition or anomaly. Instead, IT professionals must manage an application infrastructure that is a complex maze of loosely interconnected racks of servers, network components, and a multi-tiered stack of logical components including application servers, database servers, load balancers and the applications themselves. Each business process depends on a chain of components drawn from that maze, yet the components are only managed as one of a number of similar components in a rack, “farm,” or other logical silo. The result is “affordable” computing power, but at the cost of difficult-to-manage (and thus costly) system behavior.

Network management is the process of managing the various network devices and network communication links in the IT infrastructure to provide the necessary network services to the users of the network. Typical network management systems collect information regarding the operation and performance of the network and analyze the collected information to detect problems in the network. Many companies have invested in tools that do a good job of helping technical experts monitor and manage each element or silo in the multi-tiered stack of physical and logical systems. But element monitoring falls short, because when something goes wrong in the dynamically interdependent overall system, there exists no manner of knowing which physical or logical component in which rack might be the cause. In fact, there typically is not a single cause, but rather some interaction of components that really creates the problem.

Element monitoring tools are currently used to attempt to identify problems occurring in the IT infrastructure. However, the element monitoring tools in wide use in enterprises today lack a holistic view and understanding of the interdependencies of the interconnected elements of the entire IT infrastructure. There is a need to develop a system and method for managing the operating integrity of business technology systems with a comprehensive understanding of the interdependencies among all of the system components in an IT infrastructure by analyzing the end-to-end metrics and events to pinpoint the problem elements in a transaction chain.

SUMMARY

According to a feature of the disclosure, a system and method is provided for capturing and using problem fingerprints in an Information Technology (IT) infrastructure for integrity management. In one aspect, a comprehensive understanding of the interdependencies among all of the system components in an IT infrastructure is provided by analyzing the end-to-end metrics for events to pinpoint the problem elements in a transaction chain. In another aspect, the present system and method automatically captures a fingerprint of the transaction topology in an IT infrastructure at various time intervals prior to the occurrence of an event leading to a problem, such that the fingerprint provides an indication of the activity and operation of the IT infrastructure immediately preceding the problem event. The event triggering the capture of the fingerprint may be based on failures, hard threshold violations, defined transactions such as transaction slowdowns in the IT infrastructure, or user-provided occurrences.

In another aspect, the fingerprint is weighted and used to monitor IT infrastructure operation and activity to provide an indication of potential problems before the problems occur by matching real-time data on the IT infrastructure against the fingerprint to determine when the operation and activity on the IT infrastructure appears similar to activity that previously resulted in a problem event. When it is determined that there is sufficient probability that a problem event will occur based upon the real-time data matching a previously generated problem fingerprint, an alert is generated to provide sufficient notification prior to the occurrence of problem event. Based upon the comprehensive understanding of the interdependencies among all of the system components, the alert that is generated is able to specifically identify the root cause and location of the pending problem so that the alert can be delivered to a target audience. The captured problem fingerprint is device independent so that it can be applied to other transactions and devices in the IT infrastructure.

For purposes of summarizing the disclosure and the advantages achieved over the prior art, certain advantages of the disclosure have been described herein. Of course, it is to be understood that not necessarily all such advantages may be achieved in accordance with any particular embodiment of the disclosure. Thus, for example, those skilled in the art will recognize that the disclosure may be embodied or carried out in a manner that achieves or optimizes one advantage or group of advantages as taught herein without necessarily achieving other advantages as may be taught or suggested herein.

All of these embodiments are intended to be within the scope of the disclosure herein disclosed. These and other embodiments of the present disclosure will become readily apparent to those skilled in the art from the following detailed description of the preferred embodiments having reference to the attached figures, the disclosure not being limited to any particular preferred embodiment disclosed.

DRAWINGS

The above-mentioned features and objects of the present disclosure will become more apparent with reference to the following description taken in conjunction with the accompanying drawings wherein like reference numerals denote like elements and in which:

FIG. 1 is a flow diagram of a method for generating fingerprints in accordance with one embodiment of the present disclosure.

FIG. 2 is a representative transaction topology in accordance with one embodiment of the present disclosure.

FIG. 3 is a flow diagram of a method for matching fingerprints in accordance with one embodiment of the present disclosure.

FIGS. 4A-4B illustrate representative problem matrices in accordance with one embodiment of the present disclosure.

FIGS. 5A-5B are representative graphical illustrations of certain correction functions in accordance with one embodiment of the present disclosure.

FIG. 6 is a representative graphical illustration of the multiple time cut probability threshold PM in accordance with one embodiment of the present disclosure.

FIG. 7 is a representative graphical illustration relating to the probability of indicating a problem in a multi-time cut scenario in accordance with one embodiment of the present disclosure.

DETAILED DESCRIPTION

The present disclosure teaches a novel system and method for generating and using problem fingerprints in an Information Technology (IT) infrastructure for integrity management. Fingerprints are developed using a comprehensive understanding of the interdependencies among all of the system components in an IT infrastructure by analyzing the end-to-end metrics and events to pinpoint the problem elements in a transaction chain. In one aspect, the present system and method automatically captures a fingerprint of the transaction topology in an IT infrastructure at various time intervals prior to the occurrence of an event leading to a problem, such that the fingerprint provides an indication of the activity and operation of the IT infrastructure immediately preceding the problem event. The event triggering the capture of the fingerprint may be based on failures, hard threshold violations, defined transactions such as transaction slowdowns in the IT infrastructure, or user-provided occurrences.

For the purposes of this disclosure, the following terms shall be understood to possess the following meaning associated therewith:

Event: a condition that indicates an abnormal behavior in a device or a transaction (Event=Symptom+device+generation time+other information).

Symptom: the actual metric and the reason an event was generated not associated with a device or transaction (Symptom=Metric+Event Reason).

Metric: the atomic being measured on a device or a transaction, where an atomic may include any fundamental entity within a system which can be measured. A metric can consist of multiple sub items (e.g. Metric=Name+Type+SubType).

Rule: a single element describing the probability of a symptom (occurring for a given tier group and subgroup) for a specific time cut. (Rule=Time Cut+Symptom+Tier Group+Tier Subgroup+Weight).

Fingerprint: a set of rules associated with a specific tier group, tier subgroup and transaction.

Transaction: a grouping of devices within a hierarchical model consisting of tier groups and subgroups. Each device will belong to a particular tier group and tier subgroup.

Tier Group: a column within a transaction hierarchy which has a unique label (identified by the user) that separates the functionality performed by the devices within that tier group from other tier groups.

Tier Subgroup: a sub-categorization of a tier group into multiple subgroups, where the devices within a tier group can be sub-categorized based on specific functionality performed by those devices.

Alert: an event that has gone through the fingerprint process and has been identified as necessitating notification of a potential problem.

Time Cut: an integer number indicating the number of time units prior to the occurrence of a problem (the problem occurs at time cut 0). The unit time is determined by a predetermined time unit (e.g., minutes) per time cut user input.

Fingerprint Generation

Referring now to FIG. 1, an operational flow diagram for one embodiment of a method of capturing a fingerprint of a problem in information technology (IT) infrastructure is illustrated. Initially, a problem in the IT infrastructure is identified in operation 100. In one aspect, the problem is identified by identifying at least one of a failure, a hard threshold violation, a defined transaction or a user-identified occurrence in at least one device and/or transaction in the IT infrastructure. In a further aspect, the defined transaction identifying the problem may be a function of a dynamic thresholding calculation that is performed on a metric in the IT infrastructure. One such method of performing dynamic thresholding calculations is described in U.S. patent application entitled, “Self-Learning Integrity Management System and Related Methods,” filed on even date herewith and assigned to the same assignee as the present application, the contents of which are hereby incorporated by reference in its entirety. By way of example, one such dynamic thresholding determination may include a determination of a transaction slowdown on the IT infrastructure.

After a problem has been identified, information related to the problem is collected in operation 102. In one aspect, the collected information includes identifying the device(s) in the IT infrastructure where the problem is originating, where a group of devices in a transaction topology in the IT infrastructure that includes the identified problem device is also determined. Referring to the illustration of one embodiment of a transaction topology shown in FIG. 2, a transaction topology 120 of a multi-tier transaction may include a plurality of tier groups 122 that are respectively defined by the functions performed by the devices 126 in that tier group 122. For example, the different tier groups 122 could respectively identify an application server, a database server, an email server, a Citrix presentation server, etc. Each tier group 122 is preferably unique when identifying the transaction topology. A tier group may include at least one tier subgroup 124 that is a sub-grouping of devices 126 in a tier group 122. As will be described herein below, when a fingerprint match is made and a problem is predicted to occur in the IT infrastructure, the identification of the location where the problem will be occurring preferably identifies the particular tier group 122 and the tier subgroup 124 in order to pinpoint the problem.

Further information collected in operation 102 will include how far back in time the system and method should look when analyzing the activity of the IT infrastructure that led up to the occurrence of the problem by determining a number of time cuts for which to obtain information and a time interval for such time cuts. All transactions within the group of devices that involve the problem device are also identified and, for a given time cut, a set of all events for the identified transactions are collected.

With reference back to FIG. 1, once the information related to the problem has been collected, a set of rules are created for the problem in operation 104. Probabilities or weights are determined and assigned to each of the rules, as will be described in greater detail below, such that a rule describes the probability of a symptom occurring for a given group of devices will indicate a problem fingerprint for a given time cut. The collective set of rules that are created for a given problem will represent the fingerprint for such problem. A determination is made in operation 106 if the number of rules in the collected set of rules meets a certain threshold number of rules that are required to be satisfied to satisfy the fingerprint creation criteria. If the number of rules falls below the certain threshold, then it is determined that not enough data exists to generate a fingerprint and the fingerprint generation process is exited. If the number of rules describing the fingerprint meets the threshold number, then it is determined that there is enough information to create a viable fingerprint for the problem. It is then determined in operation 108 whether the fingerprint is new or whether a fingerprint already exists for the given transaction and group of devices. If the fingerprint is new, a new fingerprint is generated by operation 110 and then stored in a system database 114. If the fingerprint is already in existence, then the weights in the existing fingerprint are updated with the values in the newly generated fingerprint in operation 112, where the updated fingerprint is then stored in database 1 14.

In one aspect, the fingerprint generation process can be represented mathematically according to following equations. R _(ji)=Symptom+Tier Group+Tier Subgroup+Weight, for the j-th rule and the i-th Time Cut, N _(R)(tc _(i))=the total number of rules for time cut i, and C_(ij)=the j-th subgroup in i-th tier group, for a rule R, a number of rules N, and a tier group and tier subgroup C.

In one embodiment, the input to the system and method for capturing the fingerprints may include:

-   -   a. The device where the problem is originating (d_(o))     -   b. The tier group and subgroup of problem origination (C_(lm))     -   c. The time interval in time cuts (tc_(i))     -   d. Number of time cuts (t_(N))     -   e. Problem start time (tp)

All transactions containing the problem device d_(o) are obtained within the tier group and subgroup of problem origination (C_(lm)), where this set of transactions is referred to as T_(do). For the i-th time cut, the system and method obtains the set of all events, calculates the corresponding weights and constructs the set of rules R_(ji) for the problem. If the number of rules N_(R)(tc_(i)) describing the fingerprint is equal to or above some specified number N_(R), then there is a viable fingerprint F_(P), which will be unique for a given transaction T_(K) and tier group and subgroup of problem origination C_(lm), which can be represented by the following equation:

F_(P) = [R_(ji)_(j = 0)^(N_(R)(tc_(i))), tc_(i), tp, C_(l m), T_(k) ∈ T_(d₀)]_(i = 0)^(t_(N) − 1)

Rule Weight Determination

When probability weights are assigned to each rule, the assignment of weights to rules will take into account several variables. In one aspect, these variables include the particular time cut in which the symptom observed, whether the rule is a new rule or existing rule, whether the probability assigned is high enough to accept the rule, what fraction of devices within the tier group and subgroup contain the symptom, etc.

In one aspect, a rule weight (W) can be represented by the equation:

$W = {\left( W^{base} \right)(\gamma)\left( \frac{N_{S}}{N_{D}} \right)\left( R_{W}^{{tc}_{i}} \right)}$ where,

-   -   W^(base)=Base weight factor for new rules, where 0<W^(base)<1.     -   R_(W)=Reduction factor for existing fingerprint rules that are         not repeated when a fingerprint is recaptured, where 0<R_(W)<1.     -   W^(min)=Minimum weight allowed for a rule, where a rule is         removed from the fingerprint if the rule weight falls below this         level, further where 0<W^(min)<1.     -   Y=factor to correct for whether the symptom is from the tier         group-subgroup (Cij) of where the problem is coming from, where

$\gamma = \left\{ {{{\begin{matrix} \alpha & {C_{ij} = C_{l\; m}} \\ \beta & {C_{ij} \neq C_{l\; m}} \end{matrix}\mspace{14mu}\alpha} > {\beta\mspace{31mu} 0} < \alpha},{\beta < 1}} \right.$

-   -   N_(D)=number of devices within a specified tier group.     -   N_(S)=number of devices exhibiting symptom S within a specified         tier group.     -   tc_(i)=number of the time cut.

As can be seen from the equations, as the time cut tc_(i) increases (noting that the time domain is moving further away from the problem origination time), there is a reduction in weight probability.

For existing fingerprints, if a newly obtained rule exists within the fingerprint for the specified time cut tc_(i), then the weight (W) for that rule is increased in the fingerprint. For example, the weight can be increased according to the following equation:

$W = \frac{a + {bW}}{a + b}$

which will linearly scale W towards a maximum value of 1.0

If at the specified time cut tc_(i), an existing rule in the fingerprint is not part of the newly generated rules, then the probability of the existing rule is reduced. For example, the weight can be reduced according to the following equation: W=W·R _(W)

Any rule having a weight (W) that is reduced that falls below the minimum rule weight W_(min) is eliminated from that fingerprint for the specified time cut tc_(i).

Fingerprint Matching

In another aspect, a fingerprint matching process is implemented in which a fingerprint is used to monitor IT infrastructure operation and activity to provide an indication of a pending problem before the problem occurs by matching real-time data on the IT infrastructure against the problem fingerprint to determine when the operation and activity on the IT infrastructure appears similar to activity that previously resulted in a problem event. The fingerprint may be generated from the above-described fingerprint generation process or may otherwise be input by another source into the fingerprint matching process. When it is determined that there is sufficient probability a problem event will occur based upon the real-time data matching a previously generated problem fingerprint, an alert is generated to provide sufficient notification prior to the occurrence of problem event. Based upon the comprehensive understanding of the interdependencies among all of the system components, the alert that is generated is able to specifically identify the root cause and location of the pending problem down to the tier group and subgroup so that a targeted alert can reach the relevant audience. This provides a substantial improvement over prior “blanket” alerts that would be provided to a network supervisor whenever any type of error occurred in a system, leaving the network supervisor to figure out the cause and location of the error.

A fingerprint represents a stateless, device independent and metric-less entity that can be applied back to the relevant transaction to predict pending problems in real-time before such problems occur. This process will be referred to herein as fingerprint matching and one embodiment of which is illustrated in the operation flow diagram of FIG. 3.

Initially, the existing fingerprints are read out of the database 114 in operation 130 to obtain a set of all fingerprints (F) that apply to a transaction (T) and tier group and subgroup C_(ij). For every transaction, a symptom-problem matrix is created in operation 132, wherein this matrix will be referred to hereinafter as the problem matrix. The problem matrix is created by obtaining the set of all rules from the definition of the fingerprint fεF for a given time cut tc_(i). Each set of rules for each fingerprint f will populate a portion of the problem matrix, such that entire problem matrix is populated in this manner for the entire set of all fingerprints (F) that apply to a transaction T and tier group and subgroup C_(ij). After the problem matrix has been completed for the entire set of all fingerprints (F), the problem matrix is normalized by ensuring that for problems for which symptom-tier group information does not exist a weight of zero is assigned, thus ensuring a N×M matrix.

By way of example, the problem matrix may appear as the two dimensional matrix illustrated in FIG. 4A for a specific time cut tc_(i). The top row contains the headers for the respective columns of the problem matrix, where P_(k)-C_(ij) indicates k-th problem for i-th tier group and j-th tier subgroup. The first column contains the headers for the respective rows of the problem matrix, where S_(m)-C_(ij) indicates the m-th symptom for i-th tier group and j-th tier subgroup. This matrix for a specific time cut is two dimensional, such that the complete problem matrix taken across every time cut is a three dimensional matrix, as illustrated in FIG. 4B. The three dimensional problem matrix M (P, S, tc_(i)) can be designed as a function of the problem column P, the symptom row S and the time cuts tc_(i)).

With reference back to FIG. 3, once the problem matrix is obtained, the fingerprint matching process assembles real time events in operation 134 that involve the devices that are identified in the rules in the problem matrix.

Once the problem matrix M (P, S, tc_(i)) is obtained, it is used in one embodiment to identify the most probable problem (i.e., fingerprint) with respect to the incoming event stream. In this embodiment, the procedure for identifying a problem from the incoming events is described by the following operations. When an event is detected for a device d_(o), the set of all transactions T_(do) involving this device are obtained along with the set of all devices D that are part of all transactions T_(do). A separate thread is generated by operation 136 to observe at least a portion of the events for the set of devices D. At predetermined intervals, the thread is activated to determine fingerprint probability in operation 138 by comparing a new set of events against the problem matrix. A determination is made by operation 140 whether the probability that the symptom-tier group information indicates the specified problem by determining whether the probability meets a certain threshold, where the threshold is either determined or selected to provide a cutoff point for only those problems with a high enough probability indication of a pending problem to merit alerting a user of the pending problem. If the probability does not meet the threshold, then the thread is temporarily disabled for a period of time by operation 144 until the process returns to operation 134 to begin assembling monitoring real time events again at some later point in time. If the probability exceeds the threshold, then a problem fingerprint matching alert is generated by operation 142. This alert can serve many purposes, including but not limited to notifying a user of the pending problem along with an indication of location (tier group and subgroup) where the problem occur and at what time the problem will likely occur. The alert may also be used to institute some degree of corrective measures to prevent the problem from ever occurring.

In one embodiment, the real-time probability determination is made using the problem matrix by comparing it against the new set of events in the generated thread. For a given transaction TεT_(do), the set of all devices D_(T)

D in T is obtained, and for dεD_(T), the set E_(d) of all events corresponding to device d are obtained. For every C_(ij) in T, the total number of devices n_(ij) is obtained in addition to the number of devices m_(ij) exhibiting the symptoms S_(k). A 1×M matrix column N(S) is then constructed consisting of the values m_(ij)/n_(ij) illustrating the fraction of devices exhibiting particular symptoms. For example, the 1×M matrix column could appear similar to the following:

S₁—C₁₁ 1.0 S₁—C₃₂ 0 S₂—C₂₂ 0.5 S₃—C₁₃ 0.42 S₃—C₂₂ 1.0 S₄—C₁₁ 0 S₄—C₂₁ 0.1 S₅—C₃₁ 0 S₆—C₅₁ 0

The columns in the above matrix correspond to respective columns in the problem matrix of the transaction. Any symptom-tier information combination outside of this matrix is irrelevant as it does not indicate any known problems. Every column in the transaction matrix (representing one problem) is multiplied by the above 1×M column matrix N(S). It should be noted that this is not a matrix multiplication operation; but rather a straight element to element multiplication operation. This operation is performed for every column of the matrix for all Time Cuts tc_(i). H(P,S,tc _(i))=M(P,S,tc _(i))×N(S)

The maximum fractional difference between the columns of the H matrix and the M matrix is then computed:

${f_{r}\left( {P,{tc}_{i}} \right)} = \frac{\sum\limits_{\forall S}^{\;}{H\left( {P,S,{tc}_{i}} \right)}}{\sum\limits_{\forall S}^{\;}{M\left( {P,S,{tc}_{i}} \right)}}$

The fractional difference then needs to be scaled with respect to the total weights of the symptoms, time cut, and the number of available symptoms for the specified problem: ƒ_(total)(P,tc _(i))=(ƒ_(r)(P,tc _(i))ƒ_(w)(W _(T))−ƒ_(t)(tc _(i)))ƒ_(s)(tc _(i))+c

Where:

$W_{T} = {\sum\limits_{\forall S}^{\;}{M\left( {P,S,{tc}_{i}} \right)}}$

The equation representing the calculation of f_(total) includes several correction factors that need to be applied to f_(r) (P,tc_(i)) to obtain the true probability for the fingerprint. The correction function f_(W) (W_(T)) compensates for the lack of high weight rules. If there are a series of low weight rules that make up the fingerprint then f_(r) (P,tc_(i)) needs to be lowered to compensate for that fact. The correction function f_(t) (tc_(i)) sets a minimum threshold for probability at any time cut. Thus, as the Time Cut is increased, the total probability is reduced since the further out in time from the problem the lower the probability of those events indicate a fingerprint. The correction function f_(s) (tc_(i)) is the scaling factor for the number of symptoms indicating a fingerprint. This provides a check for high probability columns that have very few symptoms. The higher the number of symptoms, the more probable that the symptoms accurately represent the problem. Thus, if very few symptoms exist, then the probability is scaled down due to the low probability that the problem can be indicated by these few symptoms. The constant c ensures that the total probability ranges between zero and one.

For problem P, the largest value for f_(Total)(P,tc_(i)) is selected: ƒ_(Total)(P)=Max{ƒ_(Total)(P,tc _(i))}

If f_(Total)(P)>f_(min), then P is a potential problem, where f_(min) is the minimum threshold probability that is chosen as a cutoff point for problems with a high enough probability to be sent to the user as an alert. The above described procedures are repeated for all transactions in T_(do).

Correction Function f_(W)(W_(T))

The basic form of the correction function f_(W)(W_(T)) can comprise any number of defined functional forms are determined proper. In one embodiment, the desired functional form may appear as illustrated in FIG. 5A which is represented by the equation: ƒ_(W)(W _(T))=a×W _(T) ² +b×W _(T) +c

where the following boundary conditions are imposed:

f_(w)(0) = 0 f_(w)(W_(T) ≥ W_(C)) = 1 ${\frac{\mathbb{d}{f_{w}\left( W_{T} \right)}}{\mathbb{d}W_{T}}\left( {W_{T} = W_{C}} \right)} = 0$

Where W_(C) is a pre-specified specified value after which point the correction function becomes 1. Applying the boundary conditions obtains the following functional form for f_(W)(W_(T)):

$\begin{matrix} {{f_{w}\left( W_{T} \right)} = {\frac{W_{T}^{2}}{W_{C}^{2}}\left( {\frac{2W_{C}}{W_{T}} - 1} \right)}} & {W_{T} \leq W_{C}} \\ {{f_{w}\left( W_{T} \right)} = 1} & {otherwise} \end{matrix}$

Correction Function f_(t)(tc_(i))

The basic form of the correction function f_(t)(tc_(i)) can comprise any number of defined functional forms as determined proper for a particular situation. In one embodiment, the desired functional form may appear as illustrated in FIG. 5B which is further represented by the equation:

$\begin{matrix} {{f_{i}\left( {tc}_{i} \right)} = {1 - \frac{\alpha_{t}}{\beta_{t} + {tc}_{i}}}} & {{{tc}_{i} = 0},1,2,\ldots} \end{matrix}$

where the following boundary conditions for this equation are:

f_(t)(0) = f_(t)⁰ ${\frac{\mathbb{d}{f_{t}\left( {tc}_{i} \right)}}{\mathbb{d}{tc}_{i}}\left( {{tc}_{i} = 0} \right)} = {{Tan}\left( {\frac{\pi}{2}\zeta_{t}} \right)}$

where ƒ_(t) ⁰ and ζ₁ are pre-specified values which range between (0,1), where these values must be greater than zero and less than 1. ƒ_(t) ⁰ represents the starting point of the function (i.e. the base minimum probability for time cut zero for a fingerprint to be valid) and ζ₁ represents the rate of growth of the function. Applying the boundary conditions and solving for the two constants (α_(t),β_(t)) obtains:

$\beta_{t} = \frac{1 - f_{t}^{0}}{{Tan}\left( {\frac{\pi}{2}\zeta_{t}} \right)}$ α_(t) = β_(t)(1 − f_(t)⁰)

Correction Function f_(s)(tc_(i))

In one aspect, the functional form for this correction factor is exactly the same as that of f_(t)(tc_(i)), thus:

${f_{s}\left( {tc}_{i} \right)} = {1 - \frac{\alpha_{s}}{\beta_{s} + N_{s}}}$ N_(s) = 0, 1, 2, … $\beta_{s} = {{\frac{1 - f_{s}^{0}}{{Tan}\left( {\frac{\pi}{2}\zeta_{s}} \right)}\alpha_{s}} = {\beta_{s}\left( {1 - f_{s}^{0}} \right)}}$

where N_(S) is the number of symptoms indicating a problem P at tc_(i). Looking back at the main equation for probability determination: ƒ_(total)(P,tc _(i))=(ƒ_(r)(P,tc _(i))ƒ_(w)(W _(T))−ƒ_(t)(tc _(i)))ƒ_(s)(tc _(i))+c

From these, it can be seen that certain conditions exist with similar boundary conditions. In one aspect, it can be seen that the minimum value for f_(Total)(P, tc_(i))_(min) is zero and the maximum value is one. Thus, the minimum and maximum values for the probability determination can be expressed as:

f_(total)(P, tc_(i))_(min) = (f_(r)(P, tc_(i))_(min)f_(w)(W_(T))_(min) − f_(i)(tc_(i))_(max))f_(s)(tc_(i))_(min) + c f_(total)(P, tc_(i))_(max) = (f_(r)(P, tc_(i))_(max)f_(w)(W_(T))_(max) − f_(i)(tc_(i))_(max))f_(s)(tc_(i))_(max) + c

It should be noted that minimum and maximum values for the various correlation functions can be described as:

f_(r)(P, tc_(i)) = [0, 1] f_(w)(W_(T)) = [0, 1] ${f_{t}\left( {tc}_{i} \right)} = \left\lbrack {{1 - \frac{\alpha_{t}}{\beta_{t}}},1} \right\rbrack$ ${f_{s}\left( {tc}_{i} \right)} = \left\lbrack {{1 - \frac{\alpha_{s}}{\beta_{s}}},1} \right\rbrack$

Where substituting each of the above minimum and maximum values into the equations defined above yields the results: c=ƒ _(t) ⁰=ƒ_(s) ⁰=ƒ⁰

Multi-Time Cut Probability Determination

In another embodiment, the fingerprint matching process will examine probabilities across multiple time cuts to determine whether a collective number of time cuts together satisfy some criteria indicating that a problem fingerprint event is being detected. The multiple time cut probability determination is invoked in the situation where the probability for any single time cut f_(Total)(P, tc_(i)) does not meet the minimum probability threshold level f_(min) to generate a problem fingerprint alert but the probabilities for a multiple number of time cuts each exceed another specified threshold, which can also be an indicator of a fingerprint event. While it is understood that other embodiments could select non-adjacent time cuts in the multiple time cut probability determination, the multiple time cuts are preferably selected as adjacent time cuts to indicate a trend or continuity of events in the system having a probability of leading to a problem. Since the multiple time cut probability determination is examining events that occur in multiple time cuts, the probability required to indicate a problem fingerprint over multiple time cuts preferably utilizes a different multiple time cut probability threshold P_(M) that is lower than the single time minimum probability threshold level f_(min). The multiple time cut probability threshold P_(M) may be selected to be a fixed value or may alternatively vary with respect to the number of consecutive time cuts. In one aspect, the multiple time cut probability threshold P_(M) decreases as the number of consecutive time cuts indicating some probability increases. In other words, the larger the number of consecutive time cuts containing some probability of a problem existing, the smaller the threshold value required to indicate a problem fingerprint. Conversely, when consecutive time cuts possess larger probability values, fewer numbers of consecutive time cuts with such probabilities are required to indicate a problem fingerprint.

By way of example, the minimum probability threshold level f_(min) required to generate a problem fingerprint alert for a single time cut may be selected for a particular situation to be 0.75. If the probabilities for the monitored time cuts f_(Total) (P, tc_(i)) are continuously generating probabilities between 0.4-0.6, then any single time cut will not reach the minimum probability threshold level f_(min) sufficient to generate a problem fingerprint alert. However, the fact that continual probabilities between 0.4-0.6 are being reported during consecutive time cuts indicates that the monitored system is not operating ideally. Thus, a different multiple time cut probability threshold P_(M) can be selected, such as a value of P_(M)=0.3. When a certain number of consecutive time cuts possess a probability f_(Total)(P, tc_(i)) above the multiple time cut probability threshold P_(M) (e.g., 0.3 in this example), then the multiple time cut probability determination will generate a problem fingerprint alert even though the minimum probability threshold level f_(min) for any single time cut was never reached.

One embodiment of the form for the multiple time cut probability threshold P_(M) can be represented in the graphical illustration shown in FIG. 6 according to the following equation:

${P_{M}\left( \overset{\_}{\tau} \right)} = \frac{a}{{\mathbb{e}}^{b\;\overset{\_}{\tau}}}$

Where:

-   -   τ≡number of sequential time cuts for problem determination (2,         3, . . . , t_(N))     -   τ≡τ−2     -   P_(M)( τ) probability threshold above which a fingerprint is         indicated when τ adjacent time cuts exceeds its value

And the boundary conditions are:

P_(M)(0) = P_(M)⁰ ${\frac{\mathbb{d}{P_{M}(0)}}{\mathbb{d}\overset{\_}{\tau}}(0)} = {- {{Tan}\left( {\frac{\pi}{2}\zeta_{M}} \right)}}$

Such that the first and second boundary conditions lead to:

a = P_(M)⁰ $\frac{\mathbb{d}P_{M}}{\mathbb{d}\overset{\_}{\tau}} = {{{- \;\frac{{bP}_{M}^{0}}{{\mathbb{e}}^{b\;\overset{\_}{\tau}}}}->b} = \frac{{Tan}\left( {\frac{\pi}{2}\zeta_{M}} \right)}{P_{M}^{0}}}$

Where P_(M) ⁰ is the minimum probability for two adjacent time cuts to indicate a fingerprint ( τ=0) and ζ_(M) is the rate of decay of the function. This indicates that the value P_(M)( τ) required to indicate a problem gets smaller as the number of adjacent time cuts with probabilities that exceed P_(M)( τ) increases.

In one aspect, to determine whether a fingerprint is indicated through multiple time cuts, a matrix [M] is constructed which contains the information of whether the probabilities at a particular time cut exceeded P_(M)( τ). For example, the matrix [M] may appear as:

$M = \begin{pmatrix} \; & {{tc}_{i} = 0} & 1 & 2 & 3 & 4 & 5 \\ {\overset{\_}{\tau} = 0} & 0 & 1 & 0 & 0 & 0 & 0 \\ {\overset{\_}{\tau} = 1} & 1 & 0 & 1 & 1 & 1 & 0 \\ {\overset{\_}{\tau} = 2} & 1 & 1 & 1 & 0 & 0 & 1 \\ {\overset{\_}{\tau} = 3} & 0 & 1 & 1 & 0 & 1 & 1 \\ {\overset{\_}{\tau} = 4} & 0 & 1 & 1 & 1 & 1 & 1 \end{pmatrix}$

Where the rows are indexed by τ and the columns are indexed by the time cuts tc_(i). A “1” in the matrix indicates that at the given time cut tc_(i) and τ, the probability f_(Total)(P,tc _(i))>P_(M)( τ), otherwise a “0” is inserted into the matrix. Note that since:

-   -   tc_(i)=(0, 1, . . . ,t_(N)−1)     -   τ=(0, 1, . . . , t_(N)−2)         then [M] is an (t_(N)−1)×(t_(N)) matrix. To determine if a         fingerprint is indicated for a given τ, the values of the matrix         for that row are analyzed and if the sum of τ= τ+2 consecutive         columns equals τ then a fingerprint is indicated. The time cut         for that fingerprint will be the smallest tc_(i) of the summed         group. As an example, for τ=1, three consecutive columns are         required to have a value of 1. From the above matrix example,         this condition occurs for tc_(i)=2, 3, 4, thereby indicating a         fingerprint at time cut 2.

In one aspect, to determine the probability of a fingerprint in a multiple time cut probability determination, the average deviation of the difference between the computed time cut probability f_(Total)(P,tc_(i)) and P_(M)( τ) is used as a reference. This is defined as:

$\Delta \equiv {\frac{1}{\tau}{\sum\limits_{i = 1}^{\tau}\frac{{f_{total}\left( {P,{tc}_{i}} \right)} - {P_{M}\left( \overset{\_}{\tau} \right)}}{1 - {P_{M}\left( \overset{\_}{\tau} \right)}}}}$

Δ represents the average fractional deviation of the various time cuts within a group of τ adjacent time cuts, from the theoretical maximum deviation of 1−P_(M)( τ). Δ ranges from (0,1). One embodiment of a desired form of the probability represented by the equation is illustrated by the graphical illustration in FIG. 7.

The boundary conditions for the multi-time cut probability are:

$\begin{matrix} {{f_{Multi}\left( {P,\Delta} \right)} = f_{Multi}^{0}} & {{{when}\mspace{14mu}\Delta} = 0} \\ {{f_{Multi}\left( {P,\Delta} \right)} = 1} & {{{when}\mspace{14mu}\Delta} = 1} \\ {\frac{\mathbb{d}{f_{Multi}\left( {P,\Delta} \right)}}{\mathbb{d}\Delta} = 0} & {{{when}\mspace{14mu}\Delta} = 1} \end{matrix}$

where f_(Multi)(P,Δ) is the probability of indicating a problem P in a multi time cut scenario and f_(Multi) ⁰ represents the minimal probability on indicating a fingerprint for the condition that ƒ_(Total)(P,tc_(i))=P_(M)( τ) Through the use of an equation of the form: ƒ_(Multi)(P,Δ)=aΔ ² +bΔ+c

and applying boundary conditions, the probability of indicating a problem P in a multi time cut scenario f_(Muti)(P,Δ) is: ƒ_(Multi)(P,Δ)=(ƒ_(Multi) ⁰−1)Δ²+2(1ƒ−_(Multi) ⁰)Δ+ƒ_(Multi) ⁰

Where if f_(Multi)(P,Δ)>f_(min-Multi), then a problem fingerprint alert is generated.

In various embodiments, the system and method for generating and using fingerprints for integrity management is operational in an IT infrastructure or with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known computing systems, environments, and/or configurations that may be suitable for use with the invention include, but are not limited to, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, telephony systems, distributed computing environments that include any of the above systems or devices, and the like.

The system and method for generating and using fingerprints for integrity management may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The system may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices. The computer programs are stored in a memory medium or storage medium or they may be provided to a processing unit through a network or I/O bus.

In one aspect, the present system for integrity management includes at least one central processing unit (CPU) or processor. The CPU can be coupled to a memory, ROM or computer readable media containing the computer-executable instructions for generating and using fingerprints for integrity management. Computer readable media can be any available media that can be accessed by the system and includes both volatile and nonvolatile media, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory, portable memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by the fingerprint generation and matching systems. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of any of the above should also be included within the scope of computer readable media. The computer readable media may store instructions and/or data which implement all or part of the system described herein.

In one embodiment, the system and method for generating and using fingerprints for integrity management can be incorporated within J2EE and .NET based application that can be installed in any server environment, such a Windows or Linux server. In one aspect, the present system and method can act as an agentless system where no additional software is required to be installed on the monitored devices. Instead, the present system and method may collect relevant data and perform various system availability and performance tests by sending messages to the monitored systems in the form of ICMP pings, TCP/IP messages, commands over SSH terminal sessions, via Windows Management Instrumentation (WMI) methods and other known communication methods with devices in an IT infrastructure.

Many of the embodiments described herein will be directed toward integrity management of an IT infrastructure. However, it is the intention of the present inventors that the present system and method of generating and using a fingerprint can be extended to other types of systems and models.

While the apparatus and method have been described in terms of what are presently considered to be the most practical and preferred embodiments, it is to be understood that the disclosure need not be limited to the disclosed embodiments. It is intended to cover various modifications and similar arrangements included within the spirit and scope of the claims, the scope of which should be accorded the broadest interpretation so as to encompass all such modifications and similar structures. The present disclosure includes any and all embodiments of the following claims. 

1. A method of capturing a fingerprint of a problem in an information technology (IT) infrastructure, comprising: using a computer to: identifying a problem in the IT infrastructure; collecting information related to the problem; analyzing the collected information against a fingerprint creation criteria comprising a first set of rules to determine if the collected information meets a threshold number of rules for fingerprint capture; assign probability weights to a set of events that characterize a behavior of the IT infrastructure prior to the occurrence of the problem to form a second set of rules when the fingerprint creation criteria have been satisfied; and capture a fingerprint by create creating a fingerprint comprising the second set of rules if a fingerprint does not exist or updating an existing fingerprint using the second set of rules.
 2. The method of claim 1, wherein identification of the problem comprises identify at least one of a failure, a hard threshold violation, and under-identified problem.
 3. The method of claim 1, wherein the dynamic thresholding process is used to identify the problem, the dynamic thresholding determination involving a determination of a transaction slowdown on the IT infrastructure.
 4. The method of claim 1, wherein the information collecting step comprises: identifying a problem device in the IT infrastructure; determining a group of devices in the IT infrastructure that includes the problem device; determining a number of time cuts in which to obtain information and a time interval for the time cuts; identifying all transactions within the group of devices that involve the problem device; and for a given time cut, obtaining a set of all events for the identified transactions.
 5. The method of claim 4, wherein the analysis of the collected information to determine if the fingerprint creation criteria have been satisfied comprises: collecting a the first set of rules describing the probability of a symptom indicating a fingerprint for a given time cut; and determining if the number of rules in the collected first set of rules meets a certain number defined to be sufficient to satisfy the fingerprint creation criteria.
 6. A computer storage medium having program instructions stored thereon executable by a processing unit for capturing a fingerprint of a problem in information technology (IT) infrastructure by performing the steps of: identifying a problem in the IT infrastructure; collecting information related to the problem; analyzing the collected information against a fingerprint creation criteria comprising a first set of rules to determine if the collected information meets a threshold number of rules for fingerprint capture; assigning probability weights to a set of events that characterize a behavior of the IT infrastructure prior to the occurrence of the problem to form a second set of rules when the fingerprint creation criteria have been satisfied; and capturing a fingerprint by creating a fingerprint comprising the second set of rules if a fingerprint does not exist or updating an existing fingerprint using the second set of rules.
 7. The computer storage medium of claim 6, further comprising program instructions stored thereon for identifying the problem by identifying at least one of a failure, a hard threshold violation, and a user-identified problem.
 8. The computer storage medium of claim 6, wherein the dynamic thresholding process is used to identify the problem, the dynamic thresholding determination involving a determination of a transaction slowdown on the IT infrastructure.
 9. The computer storage medium of claim 6, further comprising program instructions stored thereon for collecting the problem-related information by: identifying a problem device in the IT infrastructure; determining a group of devices in the IT infrastructure that includes the problem device; determining a number of time cuts in which to obtain information and a time interval for the time cuts; identifying all transactions within the group of devices that involve the problem device; and for a given time cut, obtaining a set of all events for the identified transactions.
 10. The computer storage medium of claim 9, further comprising program instructions stored thereon for analyzing the collected information to determine if the fingerprint creation criteria have been satisfied by: collecting a the first set of rules describing the probability of a symptom indicating a fingerprint for a given time cut; and determining if the number of rules in the collected first set of rules meets a certain number defined to be sufficient to satisfy the fingerprint creation criteria.
 11. A device comprising: a fingerprint generator in an information technology (IT) infrastructure, wherein the fingerprint generator is adapted to: identify a problem in the IT infrastructure; collect information related to the problem; analyze the collected information against a fingerprint creation criteria comprising a first set of rules to determine if the collected information meets a threshold number of rules for fingerprint creature; assign probability weights to set of events that characterize a behavior of the IT infrastructure prior to the occurrence of the problem to form a second set of rules when the fingerprint creation criteria are satisfied; and capture a fingerprint by create creating a fingerprint comprising the second set rules if a fingerprint does not exist or updating an existing fingerprint using the second set of rules.
 12. The device of claim 11, wherein the problem is identified by at least one of a failure, a hard threshold violation, and a user-identified problem.
 13. The device of claim 11, wherein a dynamic thresholding process is used to identify the problem, the dynamic threshold determination involving a determination of a transaction slowdown on the IT infrastructure.
 14. The device of claim 11, wherein the device is further adapted to collect information by: identifying a problem device in the IT infrastructure; determining a group of devices in the IT infrastructure that includes the problem device; determining a number of time cuts in which to obtain information and a time interval for the time cuts; identifying al transactions within the group of devices that involve the problem device; and for a given time cut, obtaining a set of all events for the identified transactions.
 15. The device of claim 14, wherein the fingerprint creation criteria are satisfied by; collecting a the first set of rules describing the probability of a symptom indicating a fingerprint for a given time cut; and determining if the number of rules in the collected first set of rules meets a certain number defined to be sufficient to satisfy the fingerprint creation criteria.
 16. A device comprising: using a computer to identify a problem in the IT infrastructure; collect information related to the problem; analyze the collected information to determine if fingerprint creation criteria, the fingerprint creation criteria comprising a first set of rules, to determine if the collected information meets a threshold number of rules for fingerprint capture have been satisfied; assign probability weights to set of events that characterize a behavior of the IT infrastructure to form a second set of rules when the fingerprint creation criteria are satisfied; and capture a fingerprint by create creating a fingerprint comprising the second set of rules or updating an existing fingerprint using the second set of rules, wherein the second set of rules reflects a state of a transaction the IT infrastructure at a time cut prior to the occurrence of the problem.
 17. The method of claim 16, further comprising identifying the problem by identifying at least one of a failure, a hard threshold violation, and a user-identified problem.
 18. The method of claim 16, wherein a dynamic thresholding process is used to identify the problem, the dynamic thresholding determination involving a determination of a transaction slowdown on the IT infrastructure.
 19. The method of claim 16, wherein the problem-related information collecting step comprises: identifying a problem device in the IT infrastructure where the problem is originating; determining a group of devices in the IT infrastructure that includes the problem device; determining a number of time cuts in which to obtain information and a time interval for the time cuts; identifying all transactions within the group of devices that involve the problem device; and for a given time cut, obtaining a set of all events for the identified transactions.
 20. The method of claim 16, wherein the analysis of the collected information to determine if the fingerprint creation criteria have been satisfied comprises: collecting a the first set of rules describing the probability of a symptom indicating a fingerprint for a given time cut; and determining if the number of rules in the collected first set of rules meets a certain number defined to be sufficient to satisfy the fingerprint creation criteria. 